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Abstract Body 


Background / Context: 

Prior research on mentoring relationships outside of school does point toward relationship 
closeness (for example, DuBois & Neville, 1997; Parra et ah, 2002) and related indicators of the 
emotional quality of the mentor-protege tie (for example, Thomson & Zand, 2010; Zand et ah, 
2009) as important influences on youth outcomes. There is preliminary evidence that this may 
also be the case for School Based Mentoring (SBM), or at least that closeness promotes protege 
and mentor perceptions of relationship quality (Pryce & Keller, 2012). To date, however, there 
has not been a rigorous test of the role of relationship closeness in promoting academic outcomes 
in SBM. The present research addresses this gap using data from a randomized trial evaluation of 
the BBBSA SBM program conducted by Public/Private Ventures (P/PV). 

Prior work with the same data set examined the effects on academic outcomes of match length 
and status (that is, whether the original match was intact and, if not, whether the protege had 
been rematched with a new mentor). That work found negative impacts for proteges given a 
new mentor after a prior match was tenninated (Grossman, Chan, Schwartz, & Rhodes, 2012). 
Theoretically, though, a positive emotional quality or tenor in a mentoring relationship should 
mediate the benefits of program participation, independent of whatever effects may be 
associated with relationship length or status (Rhodes, 2005). Accordingly, in our analyses we 
explore whether relationship quality matters even when relationship length and current status 
are taken into account. 

Purpose / Objective / Research Question / Focus of Study: 

The overarching aim of this paper is to enrich the field’s understanding of how volunteer 
mentors can best support the academic mission of schools. 

Our central empirical analysis investigates whether emotionally closer relationships between 
mentors and proteges lead to better academic outcomes. We wish to determine whether SBM 
works primarily through the connection that the protege feels with the mentor rather than through 
other, more direct processes, such as engaging the child in academic activities during meetings. 
Outcomes include teachers’ perceptions of student academic perfonnance (Pierce, Hamm, 
&Vandell, 1999), quality of work (Herrera, 2004), completion of work (Herrera, 2004), and 
scholastic efficacy (Harter, 1985). 

Setting: 

The sample for our study consists of the students who participated in the randomized control trial 
of the Big Brothers Big Sisters of America (BBBSA) SBM program during the 2004-2005 
school year (Herrera et ah, 2007). Study participants were recruited by 10 BBBSA study 
agencies across the country, each with four or more years of experience in SBM. Recruitment 
proceeded in the usual way, that is, mostly through referrals by school staff; children had to be 
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entering grades four through nine and had to have parental consent to be part of the research. 
Over one thousand (1,139) students across 71 public schools in rural and urban school districts 
(41 elementary, 27 middle, and 3 high schools) met these criteria. 


Population / Participants / Subjects: 

Table 1 reports basic descriptive statistics for the control and treatment samples at baseline. 
Overall, just over half (54 percent) of the students were female and the majority (63 percent) 
were members of ethnic and racial minority groups. Most (69 percent) received free or reduced- 
price lunches. On average they struggled academically, receiving below a 3.0 on the teacher 
survey (the rating for “average” or “satisfactory” perfonnance). The average age was 11, 
standard deviation of 1.7. 

Intervention / Program / Practice: 

In the BBBS School-Based Mentoring (SBM) program adults or older students meet with their 
proteges on school grounds during the school day or immediately after, typically for one hour per 
week during the academic year. BBBSA SBM program has well-defined national standards. 
These include, for example, an expectation that mentors receive monthly support contacts from 
program staff. 

The amount of exposure to mentors varied by pair. Many pairs met for between 45 and 60 
minutes (40 percent) but another 39 percent met for more than an hour. Almost all matches met 
three (10 percent) or four (79 percent) times a month. Over the year, the average protege 
received 17 hours of mentoring over 5.3 months. 

The particular activities each pair engaged in also varied. About half (49 percent) of SBM 
programs had proteges meet their mentors only during the school day, with the others meeting 
after school (47 percent) or both during and after school (4 percent). Programs reported using a 
variety of locations in the school for meetings, including the cafeteria (41 percent), library (34 
percent), and a designated classroom (33 percent). Some programs used more than one meeting 
place. Almost half of all mentoring pairs shared their meeting space with other pairs. 

Research Design: 

Using results from a stratified (on the school level) Randomized Control Trial from the impact 
study of BBBS, we employ instrumental variables and other approaches to provide insight into 
why the BBBSA school-based mentoring program is effective. 

Because of course relationship closeness cannot be randomly assigned, we use instrumental 
variables and other controls to account for factors such as the preexisting characteristics of 
proteges that may contribute to both relationship closeness and youth outcomes. Two-stage 
least squares (2SLS) is used to estimate this effect of the treatment on the treated, where Zj acts 
as an instrument for T;. 
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This instrumental variables estimate can be interpreted as the average treatment effect for the 
compliers — that is, the effect on the students who fonn a close relationship with a program 
mentor — under two assumptions: monotonicity and independence (Angrist, Imbens, & Rubin, 
1996). Monotonicity requires that the assignment of a mentor increase the likelihood of a new 
close mentoring relationship for each and every student assigned to treatment. This requirement 
is met, since control students were not offered the opportunity to fonn relationships with 
BBBSA mentors. Independence requires two things: first, that we be able to identify the causal 
impact of the instrument, which is also true by construction here, and second, that potential 
outcomes not be directly affected by the instrument. We test the second requirement by taking 
advantage of the variation in program activities. 

Data Collection and Analysis: 

The students’ teachers and mentors also completed baseline surveys in the fall of 2004. Follow- 
up surveying of all groups occurred at the end of the school year, beginning in April 2005. All 
but 6 percent of the proteges completed the survey at follow up. Survey completion rates were 
slightly lower for teachers, with follow-up outcome measures missing for approximately 16 
percent of the overall sample." Importantly, however, survey non-completion rates are almost 
identical for the treatment and control groups (6 percent versus 7 percent for the student survey, 
and 16 percent versus 17 percent for the teacher survey). 

We estimate the causal effect of having a close relationship with a mentor, that is, the effect 
among the “compliers” with the SBM treatments, a la Angrist, Imbens, and Rubin (1996). In 
producing our estimates, we must take care not to reintroduce selection bias into the results. 
Although treatment status, Z,. was assigned randomly, the students who actually received the 
intended treatment, Tj, are not necessarily a random subset. These students may have an 
underlying characteristic, A;, affecting both their ability to form a relationship with a mentor 
and their academic progress over the study year — for example, a general ability to have good 
relationships with adults. One approach is to control for the characteristic explicitly. The youth 
survey includes questions about a student’s relationship with his or her teacher, and we use 
baseline responses to construct a variable A;, which provides a direct measure of a student’s 
ability to have a close relationship with a non-parental adult, to include in our ordinary least 
squares (OLS) regressions. 


Y; - CI3 + ^3Ti + pA; + X;y3 + Si ( 3 ) 


Alternatively, when compliance is imperfect, we can use the randomized treatment status, Z h as 
an instrument for the treatment of interest, having a relationship with a mentor, Tj. As 
established in Angrist, Imbens, and Rubin (1996), random assignment status is an ideal 
candidate for an instrument because it is independent of student characteristics. We thus derive 
the Wald estimate of the causal effect among compliers: the difference in outcomes between the 
nominal treatment and control groups divided by the difference in the probability of receiving 
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treatment between the groups. This estimate, which provides the effect of the treatment on the 
treated or, more generally, the local average treatment effect, measures the effect of program 
participation on those proteges who successfully connected with their assigned mentors (Bloom, 
1984; Imbens & Angrist, 1994). 

Findings / Results: 

We find evidence that a close mentoring relationship positively affects academic performance. 
Effect sizes, obtained by dividing the impact coefficients reported in the table by the standard 
deviation of the appropriate outcome measure, range from 0.13 standard deviations (for overall 
academic perfonnance and scholastic efficacy) to 0.18 standard deviations (for completeness of 
schoolwork), and are consistent across alternative specifications. 


When we interact the match closeness indicator with match status, however, we find that only 
close, intact matches have an impact. Proteges in intact matches who are not close to their 
mentors fared no better than they would have without mentors. Proteges who were rematched 
and had a close relationship with their new mentors were not negatively affected. Indeed, there 
is evidence that rematched students in close relationships were doing significantly better than 
their control peers with respect to the quality of their schoolwork, and it is possible that the 
positive impacts of being matched had not yet grown large enough to be statistically significant 
for the rematched students’ other out- comes. Thus, the current study refines the work of 
Grossman et al. (2012) to suggest that rematching does not have negative effects as long as the 
new relationship clicks. 

Conclusions: 

We find the mentor-protege relationship to be the core element of mentoring, and we believe that 
other analyses of experimental data on mentoring programs should consider the partial 
compliance model used in this paper. Although instrumental variables and other non- 
experimental techniques do not produce results as convincing as evidence from randomized 
control trials, in the end it is not possible to randomly assign close relationships. Statistical 
techniques can appropriately adjust for the presence or absence of closeness. 

Many schools have been struggling to meet increasingly higher standards and supporting at-risk 
students to stay in school and achieve academic success is a foremost priority. Under binding 
budget constraints, volunteer labor will be increasingly attractive. We find that school-based 
mentoring programs that aim to provide relationships can improve academic outcomes. Not 
every adult can be a good tutor, but many of them could help students academically by being a 
caring presence. 
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Appendix B. Tables and Figures 


Table 1: Student Characteristics at Baseline 



Control 

Treatment 

Treatment Difference 
(Control-Treatment) 


Mean 

SD 

Mean 

SD 

Mean 

SD 

Age 

11.22 

1.66 

11.24 

1.67 

-.02 

.099 

African-American/Hispanic 

(proportion) 

0.61 

0.49 

0.64 

0.48 

-.04 

.029 

Female (proportion) 

0.54 

0.50 

0.54 

0.50 

.00 

.030 

Free/reduced-price lunch 
(proportion) 

Teacher-rated outcome measures: 

0.69 

0.43 

0.69 

0.43 

-.00 

.025 

Overall academic performance 

2.47 

1.09 

2.56 

1.10 

-.08 

.070 

Quality ofschoolwork 

2.82 

0.91 

2.84 

0.94 

-.02 

.058 

Completion ofschoolwork 

2.96 

0.99 

3.00 

1.06 

-.04 

.065 

Student-reported outcome measure: 
Scholastic efficacy 

2.75 

.64 

2.80 

.62 

-.05 

.038 

Sample Size 

584 


565 
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